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PREFACE 


The  Institute  for  Defense  Analyses  (IDA)  prepared  this  document  under  a  task  titled 
“DHS  Program  Review  Training  Workshop”  for  the  Department  of  Homeland  Security 
(DHS).  The  purpose  of  the  task  was  to  provide  training  to  staff  of  the  DHS  Office  of 
Program  Analysis  and  Evaluation  (PA&E)  in  preparation  for  the  FY  2010-2014  program 
review  process.  As  an  outgrowth  of  that  work,  IDA  undertook  a  review  of  the  process  to 
assess  its  effectiveness  and  provide  recommendations  for  improvements.  This  document 
provides  that  assessment  and  recommendations. 

We  thank  Dr.  John  Whitley  of  DHS  PA&E,  who  sponsored  this  after-action  review, 
and  the  valuable  contributions  of  the  following  members  of  his  staff  Nathan  Borst,  Sarah 
Bouker,  Morgan  Geiger,  Marianna  Hennig,  Brian  LeFebvre,  Kate  Mishra,  Gregory  Pejic, 
Alexandria  Phounsavath,  and  William  R.  Reed. 

The  IDA  team  sincerely  appreciates  the  contributions  of  the  following  individuals: 
DHS  Deputy  Secretary  Paul  A.  Schneider;  DHS  Under  Secretary  for  Management  Elaine 
Duke;  DHS  Deputy  Under  Secretary  for  Mission  Integration,  Office  of  Intelligence  and 
Analysis,  James  Chaparro;  DHS  Chief  Financial  Officer  (CFO)  David  Norquist;  U.S. 
Immigration  and  Customs  Enforcement  (ICE)  Deputy  Assistant  Secretary  for 
Management  Theresa  C.  Bertucci;  ICE  Deputy  Assistant  Secretary  for  Operations  John  P. 
Torres;  U.S.  Customs  and  Border  Protection  (CBP)  Deputy  Commissioner  Jayson  P. 
Ahern;  and  CBP  Deputy  Assistant  Commissioner/Deputy  CFO  Elaine  Killoran. 

Finally,  the  IDA  technical  reviewer,  Thomas  P.  Frazier,  provided  valuable 
suggestions  for  improving  the  report. 
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A.  INTRODUCTION 


The  Department  of  Homeland  Security  (DHS),  Office  of  Program  Analysis  and 
Evaluation  (PA&E),  asked  IDA  to  follow  the  DHS  FY  2010-2014  program  review 
process,  assess  its  effectiveness,  and  provide  written  recommendations.  This  report 
documents  those  recommendations. 

IDA  staff  attended  various  meetings  related  to  the  DHS  FY  2010-2014  program 
review,  including  DHS  component  Resource  Allocation  Plan  Briefs  to  the  DHS  Program 
Review  Board,  PA&E  staff  weekly  issue  team  meetings,  program  review  issue  team 
meetings,  and  Resource  Allocation  Decision  Briefs  to  the  Program  Review  Board  to 
assess  how  well  the  process  executed.  After  the  conclusion  of  the  DHS  FY  2010-2014 
program  review,  IDA  staff  interviewed  Deputy  Secretary  Paul  A.  Schneider  and, 
individually,  seven  members  of  the  Program  Review  Board  to  solicit  comments  on  their 
experiences  with  the  program  review.  These  interviews  were  arranged  by  DHS  PA&E, 
but  members  of  that  office  were  not  present.  IDA  staff  also  interviewed  DHS  PA&E 
analysts  in  a  workshop  setting. 

Questions  were  distributed  in  advance  of  the  interviews  (reproduced  in  the 
appendix  to  this  document),  not  to  dictate  the  content  of  the  interviews,  but  to  focus 
attention  on  the  issues  of  most  importance.  Several  interviewees  provided  written 
feedback,  and  these  comments  were  incorporated  into  the  analysis. 

IDA  staff  developed  recommendations  in  the  context  of  DHS’s  Planning, 
Programming,  Budgeting,  and  Execution  (PPBE)  system  documents,  DHS  plans  and 
goals  for  the  DHS  FY  2010-2014  program  review  process,  and  IDA  staff  experience  with 
the  Department  of  Defense’s  PPBE  system. 

In  the  background  section  that  follows,  we  write  briefly  about  the  DHS  PPBE 
process.  Following  the  background  section,  we  make  observations  related  to  the  goals 
that  DHS  set  for  the  FY  2010-2014  program  review.  In  the  recommendation  section,  we 
document  our  recommendations  and  provide  the  bases  upon  which  the  recommendations 
were  formed. 

B.  BACKGROUND 

DHS  uses  a  PPBE  process  to  develop  its  Future  Years  Homeland  Security  Program 
(FYHSP),  a  strategic  plan  that  identifies  5-year  program  resource  requirements,  as  well  as 
the  Department’s  annual  President’s  Budget  request.  The  DHS  PPBE  system  establishes 
parameters  and  guidelines  for  implementing  and  executing  the  current  budget.  DHS 
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monitors  progress  and  ensures  accountability  throughout  the  PPBE  process  and,  after 
execution,  through  comprehensive  reporting. 

The  DHS  Office  of  the  Chief  Financial  Officer  (OCFO)  Financial  Management 
Policy  Manual  provides  a  background  for  PPBE,  discussing  its  history,  goals,  and 
methods.  We  drew  on  it  to  provide  the  brief  description  of  the  planning  and  programming 
phases  provided  here. 

The  planning  phase  of  the  DHS  PPBE  system  is  led  by  the  Office  of  Policy  with 
input  from  PA&E/OCFO  and  other  offices  throughout  the  DHS.  The  Integrated  Planning 
Guidance  (IPG)  is  the  final  product  of  the  planning  phase.  The  IPG  fonns  the  basis  for 
threat  infonnation  to  guide  component  resource  plan  development.  The  annual  IPG  also 
serves  as  the  strategic  framework  for  the  DHS  PPBE  programming  phase. 

The  programming  phase  is  led  by  PA&E  in  the  OCFO.  PA&E  is  also  responsible 
for  integrating  the  planning,  programming,  and  budgeting  phases  of  the  DHS  PPBE 
system.  PA&E,  in  close  coordination  with  the  Office  of  Management  and  Budget  (OMB) 
in  the  Executive  Office  of  the  President,  develops  and  issues  FYHSP  (fiscal)  guidance. 
The  fiscal  guidance  provides  fiscal  controls  for  each  DHS  component  organization  for 
each  year  of  the  5-year  plan.  PA&E  also  issues  Resource  Allocation  Plan  (RAP) 
submission  instructions  for  the  DHS  component  organizations. 

Each  DHS  component  organization  is  instructed  to  develop  a  RAP  in  accordance 
with  the  IPG,  fiscal  guidance,  and  RAP  submission  instructions.  The  component  RAP 
submissions  are  to  contain  detailed  milestones  and  perfonnance  measures  with  targets 
across  the  FYHSP.  In  addition  to  within-guidance  RAP  submissions,  component 
organizations  have  historically  submitted  over-guidance  requests.  The  RAP  submissions 
are  submitted  in  early  spring  to  PA&E,  beginning  the  annual  PA&E-led  program  review. 
PA&E  reviews  the  component  RAP  submissions  to  ensure  they  are  consistent  with  the 
fiscal  guidance,  IPG,  and  other  Departmental  guidance. 

The  Program  Review  Board  (PRB)  is  the  body  that  oversees  program  review.  The 
Deputy  Secretary  chairs  the  PRB  and  a  senior  representative  from  each  DHS  component 
organization  has  a  seat  on  the  PRB.  The  Director,  PA&E  (DPA&E)  serves  as  the 
Executive  Secretary  of  the  PRB. 

Each  component  organization  briefs  the  PRB  on  its  resource  allocation  plans. 
During  these  meetings,  the  Deputy  Secretary  and  the  PRB  members  begin  to  identify 
issues  for  program  and  budget  review.  The  CFO  and  DPA&E  assist  the  Deputy  Secretary 
in  deciding  which  issues  will  be  selected  for  review,  and  whether  the  issue  is  better 
handled  by  PA&E  for  programmatic  issues  or  budget  for  budgeting  issues.  The  budget 
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review  is  executed  in  parallel  with  the  program  review  as  depicted  in  Figure  1.  The 
budgeting  phase  is  led  by  the  Budget  Division,  also  within  OCFO. 


Source:  “FY10-14  Program  Review,  Program  Review  Process  and  Issues,”  briefing,  22  April  2008. 

Figure  1.  FY  2010-2014  Program  Review  Schedule 


The  issues  directed  to  program  review  are  placed  in  one  of  several  functional 
categories  each  of  which  is  addressed  by  an  issue  team  that  is  led  by  a  PA&E  analyst.  On 
occasion,  some  smaller  issues  may  be  identified  as  Minor  Issues  at  the  PRB  and  tasked  to 
a  PA&E  analyst  for  review. 

The  goal  of  the  issue  teams  is  to  produce  distinct  programmatic  options  to  achieve 
defined  end-states  over  the  FYHSP.  All  organizations  with  membership  on  the  PRB  are 
invited  members  of  the  program  review  issue  teams.  Other  offices  and  subject  matter 
experts  may  be  invited  at  the  discretion  of  PA&E.  The  issue  team  develops  and  analyzes 
programmatic  alternatives  and  documents  its  results  in  a  Program  Review  Decision  Brief, 
which  is  presented  to  the  PRB  for  decision.  The  PRB  is  not  a  voting  body.  The  PRB 
informs  the  ultimate  decision,  which  is  made  by  the  Deputy  Secretary. 

The  Deputy  Secretary  issues  guidance,  which  is  documented  in  a  Draft  Resource 
Allocation  Decision  (RAD)  document.  PA&E  issues  the  Draft  RADs  individually  to 
component  organizations  for  their  review  and  a  final  chance  to  appeal  the  decision.  The 
final  RAD  is  issued  by  the  Secretary  and  Deputy  Secretary  through  PA&E.  The  final 
RAD  defines  the  DFIS  FYFISP  for  submission  to  OMB. 
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C.  OBSERVATIONS  ON  THE  DHS  GOALS  FOR  THE  FY  2010-2014 
PROGRAM  REVIEW 

This  section  discusses  IDA  observations  related  to  the  goals  that  DHS  set  for  the 
FY  2010-2014  program  review.  Following  this  section  are  IDA  recommendations  for  the 
DHS  PPBE  process. 

DHS  Deputy  Secretary  set  the  following  goals  for  the  FY  2010-2014  program 
review: 

•  Facilitate,  in  conjunction  with  the  budget  review,  production  of  a  prioritized, 
balanced  DHS  FY  2010-2014  FYHSP  within-guidance  levels  and  a 
consolidated,  prioritized  over-guidance  package. 

•  Continue  building  and  institutionalizing  a  collaborative,  transparent  program 
review  process  as  part  of  the  DHS  PPBE  process. 

•  Promote  the  transfer  of  information  and  continuity  in  the  resource  allocation 
process  during  the  change  in  administration. 

The  DHS  program  review  process  produced  a  DHS  FY  2010-2014  FYHSP  at  the 
within-guidance  levels  and  a  consolidated,  prioritized  over-guidance  package  that  was 
submitted  to  OMB.  Every  member  of  the  Program  Review  Board  was  provided  the 
opportunity  to  learn  about  each  issue  and  assign  a  priority.  PA&E  combined  the  assigned 
priorities  to  provide  a  ranking  from  1  to  n,  the  total  number  of  issues.  In  coordination 
with  the  Deputy  Secretary  and  the  CFO,  boundary  lines  were  drawn  classifying  the  over¬ 
guidance  requests  into  three  tiers. 

DHS  also  made  great  strides  in  educating  DHS  leadership  and  analysts  on  the  issues 
most  likely  to  be  revisited  by  the  new  administration.  Many  PRB  members  interviewed 
felt  the  FY  2010-2014  program  review  helped  to  prepare  them  to  answer  questions  that 
may  be  asked  by  the  new  administration. 

The  PRB  members  generally  concluded  that  DHS  executed  a  collaborative, 
transparent  program  review  process  as  part  of  the  DHS  PPBE  system.  DHS  adopted  a 
program  review  process  to  develop  the  FYHSP  and  shared  it  in  Chapter  2  of  the  OCFO 
Financial  Management  Policy  Manual.  The  participation,  activities,  and  schedules 
discussed  in  these  documents  were  followed  for  the  most  part;  e.g.,  guidance  for  the 
component  RAP  submissions  were  issued;  the  PRB  met  to  hear  the  component  RAP 
submissions;  program  and  budget  review  issues  were  identified  and  worked  by  issue 
teams;  and  program  review  issues  were  briefed  to  the  PRB  for  decision.  Another  example 
of  the  collaborative,  transparent  process  was  manifested  at  the  PRB  meetings  where  each 
component  organization  had  a  chance  to  brief  its  base  budget  and  over-guidance  requests 
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to  all  DHS  component  organizations,  in  addition  to  the  Deputy  Secretary.  In  the 
preceding  year,  each  component  organization  briefed  the  Deputy  Secretary  alone. 

While  the  goal  to  achieve  a  transparent  process  was  largely  a  success,  the  following 
items  were  perceived  to  violate  the  transparency  policy  goal:  final  RADs  were  released 
only  to  their  respective  component  organization,  the  U.S.  Coast  Guard  (USCG)  did  not 
participate  in  the  issue  team  process  (instead  following  the  process  of  previous  years  to 
develop  its  budget),  and  decisions  on  issues  identified  as  “Quick  Action  Items”  were 
incorporated  toward  the  end  of  program  review. 

D.  RECOMMENDATIONS 

Observation  of  the  PPBE  process  and  discussions  with  participants  led  us  to  the 
following  recommendations  for  improvement.  Each  recommendation  is  followed  by  a 
discussion  of  the  observations  and  comments  that  led  us  to  formulate  the 
recommendation.  We  present  the  most  difficult  recommendations  to  implement  first.  The 
later  recommendations  streamline  the  current  process  freeing  up  analysts’  time  while 
minimizing  confusion. 

Recommendation  1  (Evolution  of  the  Programming  Process):  DHS  should 
continue  its  current  course  of  improvement  to  the  program  review  process  and  DHS 
leadership  should  enforce  compliance  with  the  process. 

DHS’s  component  organizations  are  used  to  having  a  new  DHS  headquarters 
budget  process  every  year  and  many  are  not  familiar  with  future  year  planning  and 
programming.  Further,  they  have  little  motivation  to  comply  with  parts  of  the  current 
process  they  think  are  not  in  their  best  interest.  For  example,  the  USCG  worked  off  line 
with  the  Deputy  Secretary  on  its  base  budget  and  over-guidance  requests. 

This  year,  however,  DHS  adopted  a  structured  process  for  program  review  in  which 
all  components  saw  all  of  the  issues  and  the  programmatic  alternatives  considered  by  the 
Secretary  and  Deputy  Secretary.  Some  justified  criticisms  of  the  process  were  voiced  (for 
example,  too  many  small  issues  at  the  PRB  Decision  Meeting  and  insufficient  staff  for 
the  workload  created  by  the  review),  but  the  component  organizations  unanimously 
agreed  that  this  year’s  program  review  was  (1)  better  than  any  that  went  before,  (2) 
provided  useful  information  for  future  planning  and  programming  activities,  and  (3) 
provided  an  adequate  structure  for  the  program  review  part  of  PPBE.  For  those  reasons,  it 
is  sensible  for  DHS  to  retain  the  PPBE,  continue  to  evolve  it,  and  look  to  leadership 
emphasis  and  the  continuity  of  the  process  to  improve  participation  and  cooperation. 
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Recommendation  2  (Programming  Effort):  DHS  HQ  programming  efforts  should 
focus  on  rationalization  of  capabilities  across  components  and  agencies  and  should  not 
be  restricted  to  over-guidance  requests.  The  DHS  Headquarters  and  component 
organizations  should  provide  realistic  cost  estimates  to  support  the  funding  requests  for 
the  base  budgets  in  addition  to  the  over-guidance  requests.  PA&E  should  ensure  that 
analyses  provided  with  decision  options  relate  DHS’s  programmatic  goals  to  the  funding 
provided  to  archive  those  goals.  Furthermore,  the  analyses  should  clearly  link  failure  to 
achieve  programmatic  goals  to  the  lack  of  sufficient  funding. 

No  substantial  changes  to  base  funding  levels  came  out  of  the  DHS  FY20 10-20 14 
program  review.  In  addition,  while  many  component  organizations  and  the  CFO  felt  they 
did  not  have  a  realistic  base  budget  to  start,  the  base  budget  was  hardly  addressed  during 
program  review. 

The  lack  of  activity  in  this  area  can  be  attributed  to  many  circumstances,  including 
the  change  in  administration,  implementation  of  a  new  program  review  process,  time 
constraints,  limited  in-house  analytic  capability  compared  to  the  magnitude  and  scope  of 
the  issues,  and  the  program  review  goal  that  limited  the  scope  of  program  review  to 
prioritizing  over-guidance  requests. 

All  of  these  circumstances  limited  the  ability  of  PA&E  to  get  into  the  real 
programming  issues  DHS  faces — the  cross-cutting  issues  in  the  component’s  base 
budgets.  For  example,  this  year,  the  border  control  issue  team  focused  on  the  U.S. 
Customs  and  Border  Protection  (CBP)  solution  to  border  control,  but  several  other 
component  organizations  affect  border  control,  e.g.,  the  USCG  and  ICE.  An  improved 
program  review  would  examine  the  base  budgets  for  all  component  organizations 
participating  in  and  affected  by  border  control  issues  and  allocate  resources  to  achieve  the 
best  outcome  for  the  Department  and  the  country  (or  at  least  make  progress  toward  doing 
this). 

PA&E  analysts  did  not  have  much  opportunity  to  engage  in  programming  activities 
during  this  program  review.  DHS  PA&E  sorted  over-guidance  requests,  grouped  them 
into  categories,  and  assigned  them  to  a  program  review  issue  team  or  budget  review.  The 
issue  teams  were  allocated  too  many  loosely  related  resource  requests  to  permit  effective 
indepth  analysis  as  the  basis  for  their  proposed  program  decision  options.  A  few  PA&E 
analysts  reported  that  they  spent  most  of  their  time  packaging  information  into  a  briefing 
format  for  the  Program  Review  Board. 

Recommendation  3  (Compliance  with  Guidance):  Formalize  a  PA&E  review  of 
RAP  submissions  for  compliance  with  guidance.  PA&E ’s  assessment  of  failure  to  comply 
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should  hold  up  consideration  of  over-guidance  requests  for  program  review  issue 
identification. 

There  appeared  to  be  no  formal  process  for  the  PA&E  analysts  to  review  the  RAP 
submissions  against  guidance  documents  (IPG,  fiscal  guidance,  instructions  for  RAP 
content)  and  act  upon  the  results  of  the  review.  Consequently,  some  RAP  submissions 
were  used  as  the  basis  for  program  review  without  key  critical  information  such  as 
performance  metrics,  priority  ranking,  and  accompanying  exhibits.  Issue  teams  spent 
precious  and  limited  time  identifying,  collecting,  and  developing  information  that  should 
have  been  available  from  the  beginning.  A  reasonable  approach  for  limiting  the  number 
of  program  review  issues  was  not  followed. 

The  DHS  program  review  process  does  not  have  the  tools  for  enforcing  fiscal 
constraint/restraint.  If  PA&E  can  hold  up  a  request  for  over-guidance  funds  for  non- 
compliance  of  the  RAP  submission  with  policy,  congressional  direction,  fiscal  guidance, 
planning  guidance,  and  program  review  decisions  from  the  previous  year,  it  will  increase 
the  office’s  ability  to  incorporate  proper  programming  techniques  in  the  development  of 
the  base  budgets  of  future  submissions. 

Recommendation  4  (Interface  Between  Program  and  Budget  Review):  DHS 

Policy,  PA&E,  and  Budget  need  to  discuss  and  reallocate  their  activities  regarding 
planning,  programming,  and  budgeting  so  they  are  consistent  with  DHS  documentation 
of  the  PPBE  process.  PA&E  should  focus  on  base  funding  of  cross-functional  issues  in 
the  programming  years  of  the  FYHSP.  PA&E  and  the  Budget  Office  should  co-develop 
the  interfacing  process  between  program  and  budget  review. 

As  practiced  this  year,  the  distinction  between  programming  and  budgeting  was 
vague.  According  to  the  CFO,  program  and  budget  review  shared  the  same  process — the 
only  distinction  he  made  was  the  criterion  that  sent  cross-functional  and/or  multi-year 
issues  to  PA&E  analysts  and  single-year  requests  for  FY  2010  funds  to  the 
budget  analysts. 

It  appears  that  some  PA&E  and  Budget  Office  analysts  do  not  understand  what  the 
other  is  trying  to  accomplish.  At  one  extreme,  a  PA&E  analyst  reported  that  Budget 
analysts  do  not  have  a  documented  process  for  budget  review.  However,  we  spoke  to  a 
Budget  analyst  who  clearly  outlined  a  budget  review  process  that  made  sense  and  did  not 
understand  why  Budget  analysts  would  be  perceived  as  not  having  a  process. 

Some  program  review  participants  perceived  a  lack  of  communication  between 
PA&E  and  the  Budget  side  of  the  OCFO.  It  did  not  appear  there  was  a  coordinated 
transmission  of  the  Deputy  Secretary’s  decisions  from  PA&E  to  Budget.  In  addition, 
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some  PA&E  analysts  felt  they  did  not  have  the  needed  visibility  into  budget  analysis  and 
recommended  actions. 

Recommendation  5  (Issue  Identification):  PA&E  and  the  Budget  Office  should 
develop  an  integrated  program  and  budget  review  issue  identification  and  classification 
process.  The  issue  identification  and  classification  process  should  draw  upon  RAP 
submissions  that  have  passed  the  review  for  compliance  to  guidance.  The  issue 
identification  process  should  have  criteria  that  limit  the  number  and  scope  of  the  major 
program  review  issues  that  are  handled  by  the  issue  teams.  For  example,  they  should  be 
important  enough  to  merit  the  attention;  they  should  relate  funding  to  measures  of 
success;  and  they  should  address  issues  of  compliance  with  guidance.  PA&E  analysts 
should  be  able  to  ensure  the  base  and  alternative  programs  have  a  reasonably  accurate 
cost  estimate  and  to  provide  objective  assessmen  ts  and  appraisals  of  tradeoffs. 

There  is  no  method  to  classify  an  issue  as  not  worthy  or  mature  enough  to  be 
considered  for  any  type  of  review  or  action  or  to  dismiss  an  issue  in  the  current  year  for 
consideration  of  additional  funds.  Everything  was  carried  through  and  ranked  by  the 
Program  Review  Board.  In  addition,  some  issues  that  PA&E  classified  this  year  as  “no 
further  review”  were  passed  on  for  budget  review  and  experienced  favorable  outcomes. 
Disparate  and  incomplete  handling  of  over-guidance  requests  undennines  attempts  to 
impose  fiscal  discipline  across  DHS  component  organizations  that  submit  over¬ 
guidance  requests. 

The  program  review  was  weighed  down  by  too  many  small  issues.  This  meant  that 
issues  were  handled  too  quickly,  which  hurt  PA&E  credibility. 

Some  component  participants  had  difficulty  fitting  their  issues  into  the  PA&E- 
defined  buckets.  PA&E  lost  a  few  weeks  of  review  time  “bucketing”  issues,  and  the  PRB 
did  not  have  any  real  input  into  issue  formation.  PA&E  analysts  should  have  spent  more 
time  up  front  reading  program  decision  options  to  develop  the  issues. 

Recommendation  6  (Issue  Categories):  Modify  the  issue  identification  process  to 
distinguish  the  following  three  categories  for  program  review  and  have  a  different 
process  for  each  that  allots  an  appropriate  amount  of  program  review  resources  to  the 
resolution: 

•  Minor — administrative  matters,  small  amounts  of  resources,  and  single  agency 
participation  (even  if  it  is  a  considerable  funding  request).  Minor  issues  can  be 
worked  by  few  people  and  documented  in  a  short  issue  paper.  These  issues 
would  not  consume  time  at  the  PRB  Decision  Meetings. 
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•  Directed — issues  identified  by  senior  leadership  and,  therefore,  treated  as 
major  issues.  This  process  would  need  to  adapt  to  the  needs  of  senior 
leadership. 

•  Other  Major — large  cost  or  widespread  effects  that  involve  multiple  agencies. 
Major  issues  are  the  subject  for  full  issue  team  participation  as  currently 
adopted  by  DHS  PA&E. 

The  importance  of  the  program  review  issues  that  were  identified  by  the  Deputy 
Secretary  this  year  (tenned  “quick-action  items”  by  PA&E)  may  have  been  underrated  by 
PA&E.  These  issues  were  of  special  importance  to  the  Deputy  Secretary,  and  he  was 
pleased  with  the  way  these  issues  were  handled  by  PA&E  and  the  outcomes. 

Recommendation  7  (Issue  Handling):  Modify  the  program  review  process  to 
incorporate  separate  handling  of  minor  and  directed  issues.  For  example,  minor  issues 
can  be  assigned  to  a  PA&E  or  budget  analyst  who  would  then  collect  information  and 
write  a  short  issue  paper  with  recommendations.  The  paper  could  be  posted  on  a  Web 
site  to  maintain  the  integrity  of  the  transparent  process  policy.  The  process  for  handling 
directed  issues  should  be  tailorable  to  the  wishes  of  the  current  Deputy  Secretary  on 
content,  attendance,  and  frequency  without  appearing  to  bypass  the  PRB.  To  the  extent 
possible,  directed  issues  should  be  subject  to  the  full  PRB  process. 

Some  program  review  participants  felt  they  should  have  been  included  in  the 
deliberations  of  some  of  the  quick-action  items. 

Issues  identified  as  quick-action  items  were  handled  outside  the  documented  PA&E 
program  review  process,  even  though  PA&E  management  and  analysts  participated  in  the 
analysis  of  these  issues.  These  items  were  inserted  back  into  the  PA&E  program  review 
process  during  or  after  the  Program  Review  Decision  Briefs  were  presented  to  the 
Program  Review  Board.  Some  people  felt  this  situation  was  not  in  the  spirit  of  the  goal  to 
have  an  open  and  transparent  process  and  that  they  should  have  participated  in  the 
discussions  but  were  not  included. 

Recommendation  8  (Issue  Teams):  Minor  issues  should  not  be  handled  by 
issue  teams. 

Each  PA&E  analyst  headed  an  issue  team  and  all  multi-year  over-guidance  requests 
were  distributed  to  one  of  the  issue  teams  or  to  budget  review.  Too  many  topics  were 
assigned  to  each  issue  team. 

One  PA&E  analyst  felt  their  issue  team  did  not  have  time  to  do  any  sort  of  technical 
analysis.  That  person  also  added  that  they  did  not  think  any  of  the  teams  did  any  sort  of 
original  analysis  beyond  simple  scaling. 
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Another  PA&E  analyst  wrote: 

Virtually  all  of  the  major  topics  our  teams  examined  required  some  sort  of 
operations  research  or  systems  analysis.  PA&E  currently  does  not  have  the  in- 
house  capability  to  do  this  in  the  fall  to  prepare  for  the  approaching  Program 
Review.  The  results  are  presentations  of  superficial  financial  analysis  that  basically 
re -packages  the  over-guidance  requests  more  neatly.  We  didn’t  have  any  solid 
analytical  studies  to  support  recommendations  for  true  programmatic  changes.  We 
can  only  say  if  something  is  generally  working  or  not,  we  can’t  say  how  to  fix  it. 

Some  component  executives  felt  they  did  not  have  the  manpower  to  adequately 
staff  all  issue  teams. 

Removing  the  minor  issues  from  the  responsibility  of  issue  teams  would  allow  more 
analysis  of  the  directed  and  other  major  issues. 

Recommendation  9  (Time  Slots  for  Decision  Briefs):  Time  slots  for  decision  briefs 
should  be  allocated  to  major  issues  and  tailored  to  give  more  time  to  more  important 
issues.  Work  minor  issues  outside  the  PRB  Decision  Meetings. 

This  year  major  issue  decision  briefs  were  allotted  60  minutes  in  front  of  the 
Program  Review  Board  and  minor  issues  were  allotted  15  minutes.  Some  interviewees 
thought  minor  issues  could  be  handled  offline  rather  than  going  before  the  Program 
Review  Board. 

Recommendation  10  (Distribution  of  Draft  PRB  Briefs):  The  process  should 
provide  for  earlier  distribution  of  draft  PRB  briefs  to  component  leadership  for  review. 

This  year  the  issue  teams  distributed  draft  PRB  briefs  at  the  staff  level  2  days  before 
the  decision  meeting  and  sometimes  sent  informal  read-ahead  information  a  week  in 
advance.  Despite  this,  some  PRB  members  reported  not  having  enough  time  to  prepare 
for  the  decision  meeting. 

Recommendation  11  (Pre-Program  Review  Analysis):  PA&E  should  begin 
analysis  of  cross-cutting  issues  in  the  spring  and  incorporate  a  pre-program  review 
assessment  on  issues  that  need  extra  vetting  time. 

A  few  cross-cutting  issues  required  more  time  and  different  resources  than  the 
program  review  process  permits.  For  example,  multiple  organizations  contribute  to 
border  control  (e.g.,  CBP,  USCG  and  ICE)  and  developing  an  efficient  and  integrated 
solution  to  border  control  cannot  happen  during  short,  intense  annual  program  review. 

Recommendation  12  (Program  Review  Issue  Papers):  Modify  the  process  so  that 
in  future  years  a  Program  Review  Issue  Paper  that  documents  the  analysis  that  went  into 
the  alternatives  presented  in  the  decision  brief  is  developed  in  parallel  with  the  brief  and 
is  available  to  members  of  the  PRB  before  the  decision  brief  is  presen  ted  to  the  PRB. 
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The  program  review  process  provided  PRB  briefs  for  major  issues  and  issue  papers 
for  minor  issues,  but  no  formal  documentation  of  the  analyses.  While  PA&E  intended  to 
document  its  analysis  in  the  fall  of  2008  to  prepare  for  the  new  administration,  the  best 
time  to  document  analysis  is  when  you  are  perfonning  the  analysis. 

Recommendation  13  (Status  on  RAP  submissions):  Include  the  result  of  PA&E’s 
review  of  the  RAP  submissions  against  guidance  in  the  regular  status  reports  to  the  PRB 
during  the  RAP  briefings. 

There  was  a  general  lack  of  understanding  and  awareness  of  the  importance  of 
complying  with  guidance  in  RAP  submission.  There  was  no  mechanism  for  PA&E  to 
provide  feedback  on  how  component  organizations  were  complying  with  guidance. 

Recommendation  14  (Document  Templates):  Take  the  best  of  this  year’s 
documents  and  use  them  to  make  templates  for  next  year. 

Different  presentation  and  documentation  formats  introduced  confusion  and 
increased  the  time  needed  to  understand  the  issues. 

Recommendation  15  (Integrated  Database):  Assure  that  the  RAD  from  program 
review  and  the  Schedule  B  from  the  budget  review  are  built  from  one  single  database. 
Develop  a  single  document  or  database  to  manage  all  resource  allocation  decisions.  The 
RADs  should  also  be  part  of  a  single  document  linked  to  the  management  tool. 

Configuration  management  and  configuration  control  of  programming  and 
budgeting  data  were  inadequate.  PA&E  analysts  invested  considerable  time  in 
administrative  tasks  instead  of  thinking  about  and  writing  the  RADs. 

Recommendation  16  (PRB  Meeting  Minutes):  Publish  the  Deputy  Secretary’s 
decisions  after  each  PRB  meeting  in  the  interest  of  providing  the  Department  with  a 
transparent  program  review  process  and  to  improve  communication  between  the 
program  and  budget  reviews. 

Several  instances  were  observed  where  people  had  different  interpretations  of  a 
decision  made  by  the  Deputy  Secretary  and  felt  that  the  decision  was  reversed  by  the 
budget  review. 

E.  CONCLUSIONS 

The  Department  of  Homeland  Security  has  made  significant  progress  in 
institutionalizing  a  Planning,  Programming,  Budgeting,  and  Execution  process.  This  year, 
the  efforts  of  the  program  review  issue  teams  and  the  Program  Review  Board  were 
focused  on  the  preparation  of  a  prioritized,  over-guidance  request  to  submit  to  OMB  and 
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to  prepare  the  senior  leadership  of  each  Program  Review  Board  member  component  for 
the  new  administration.  While  these  are  important  tasks,  they  are  more  limited  than  those 
of  a  classic  PPBE  process.  Consideration  of  cross-component  trades  among  the  highest 
priority  programs  and  initiatives  within  guidance  levels  would  lead  to  an  improved 
program  review  process. 
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APPENDIX:  DISCUSSION  QUESTIONS 


QUESTIONS  FOR  DEPUTY  SECRETARY,  PAUL  A.  SCHNEIDER 

1 .  The  issues  teams  analyzed  People  Screening,  Cargo  Screening,  Interior  Immigration 
Enforcement,  Facilities  Consolidation,  Biowatch,  IT,  Intelligence,  and  Border 
Security.  Was  this  the  right  set  of  issues  given  the  operating  environment  the 
Department  faces  and  pending  change  of  Administration?  How  should  issue 
determination  be  made  in  the  future? 

2.  Were  the  PRB  decision  briefings  informative  and  provide  the  infonnation  in  the  best 
way  for  you  to  make  decisions?  How  would  you  have  changed  the  decision  briefs? 

3.  Did  the  program  and  budget  review  process  give  you  the  information  you  needed, 
when  you  needed  it,  to  make  informed  resource  decisions? 

4.  Did  the  Program  Review  Board  (PRB)  discussion  and  prioritization  help  you  make 
decisions?  What  would  you  change  about  the  PRB  make-up  or  progress? 

5.  Were  the  quick  action  items  (e.g.,  USCG  acquisition,  fusion  centers,  cyber  security) 
and  minor  issues  (e.g.,  pandemic  influenza,  counter-IED)  helpful?  How  could  PA&E 
better  manage  quick  action  items  and  minor  issues  to  ensure  integration  into  the 
process  and  retain  transparency? 

6.  Was  the  process  helpful  for  transition  preparation?  Should  the  program  review 
packages/decision  briefs  be  used  as  part  of  the  transition  material? 

7.  Should  the  PRB  meet  more  regularly  during  the  year  to  discuss  resource  allocation 
issues  and  reduce  the  concentration  of  workload  during  the  program  review  period? 
Should  the  PRB  membership  after  the  transition  remain  as  deputy  component  heads 
or  be  transferred  to  component  heads? 

8.  What  would  you  change  about  the  overall  process? 

9.  Do  you  have  any  other  comments? 
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QUESTIONS  FOR  PRB  MEMBERS 

Introduction:  Mr.  Schneider’s  priorities  for  the  FY  2010-2014  program  review 

were  to: 

•  Produce  a  balanced  FY  2010-2014  Future  Years  Homeland  Security  Program 
(FYHSP)  and  FY  2010  budget  within  guidance  funding  levels  from  OMB,  and  a 
limited  prioritized  over-guidance  request  containing  key  programs  and 
initiatives  that  cannot  be  funded  within  that  level. 

•  Build  and  institutionalize  a  collaborative  transparent  process  to  support  DHS 
planning,  programming  budgeting  and  execution. 

•  Promote  continuity  in  the  resource  allocation  process  during  the  change  in 
administration. 

1.  The  first  priority  was  building  a  balanced,  prioritized  program  and  budget  with  a 
limited  over-guidance  request.  How  well  do  you  think  this  was  accomplished? 

2.  DHS  has  never  submitted  a  budget  within  the  OMB  guidance  values.  Submitting 
budgets  above  guidance  transfers  decisionmaking  from  DHS  to  OMB,  giving  them 
more  control  of  our  destiny.  How  could  more  fiscal  discipline  be  accomplished  in  the 
resource  allocation  process? 

3.  The  second  objective  was  to  institutionalize  an  analytically  based,  transparent 
resource  allocation  process.  How  well  was  this  accomplished? 

4.  Were  the  PRB  decision  briefs  infonnative  and  provide  the  information  in  the  best 
way  for  you  to  make  recommendations  to  the  Deputy  Secretary?  Was  the  analytical 
content  of  sufficient  depth?  How  would  you  have  changed  the  decision  briefs? 

5.  Did  the  Program  Review  Board  (PRB)  meetings  provide  transparency  in  the  resource 
allocation  process? 

6.  Did  the  staff-level  issue  teams  provide  transparency  in  the  resource  allocation 
process?  Did  your  staff  attend  the  issue  team  meetings?  Did  you  interact  and  get 
briefs  from  your  staff  attending  the  issue  team  meetings  while  they  were  on-going? 
Next  year  will  your  staff  participate  in  more  issue  teams? 

7.  The  issues  teams  analyzed  People  Screening,  Cargo  Screening,  Interior  Immigration 
Enforcement,  Facilities  Consolidation,  Biowatch,  IT,  Intelligence,  and  Border 
Security.  Were  the  issues  the  correct  set  of  issues  that  the  Department  faces?  How  can 
issue  selection  be  improved? 
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8.  Were  the  quick  action  items  and  minor  issues  (for  example,  Pandemic  Flu,  Counter- 
IED)  helpful?  How  could  PA&E  better  manage  quick  and  minor  issues  to  ensure 
integration  into  the  process  and  retain  transparency? 

9.  Most  of  the  discussion  in  the  FY  2010-2014  cycle  concerned  over-guidance  requests. 
In  future  years  as  the  high  growth  rates  the  Department  has  experienced  begin  to 
diminish,  the  discussion  will  begin  to  focus  more  and  more  on  the  base  and  on  trade¬ 
offs  across  component  organizations.  What  challenges  will  be  introduced  by  this  and 
how  can  these  be  overcome? 

10.  Should  the  PRB  continue  meeting  periodically  throughout  the  year  to  discuss 
resource-allocation  issues,  prepare  for  the  next  cycle,  and  try  to  smooth  the  workload 
more  evenly  throughout  the  year? 

11.  Was  the  process  helpful  for  transition  preparation?  Should  the  program  review 
packages/decision  briefs  be  used  as  part  of  the  transition  material? 

12.  Do  you  have  any  other  comments? 

QUESTIONS  FOR  PA&E  ANALYSTS 

(1)  Process  management,  e.g.,  Program  Review  Board  (PRB),  Meetings  with  the  Deputy 

Secretary  (S2),  Appeals 

•  What  important  programmatic  issues  were  not  adequately  discussed  during  the 
FY  2010-2014  program  review? 

•  What  important  programmatic  issues  were  misunderstood  by  the  PRB? 

•  How  well  did  the  program  review  process  document  the  analyses  performed  to 
support  the  decisions  made  during  this  year’s  program  review?  Will  these 
products  support  reexamination  of  the  FY  2010  budget  and  the  FY  2010-2014 
FYHSP  by  the  new  administration? 

•  What  can  be  done  to  reduce  the  number  of  appeals? 

•  What  issues  were  addressed  outside  the  documented  process  (e.g.,  end-run¬ 
arounds)? 

•  What  can  be  done  to  improve  component  participation  in  next  year’s  program 
review? 

•  How  did  leadership  support  PA&E  administration  of  the  program  review 
process?  How  can  this  be  improved  next  year? 

(2)  Program  review  issue  selection,  e.g.,  scope,  quick  action  items,  staying  on  schedule 

•  Were  the  issues  selected  the  correct  set  of  issues  for  program  review?  How  can 
PA&E  ensure  better  and  timelier  selection  in  the  future? 

•  What  tools  were  available  for  denying  the  over-guidance  requests  that  did  not 
relate  resource  requests  to  outputs/outcomes/end  states? 
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•  What  could  be  improved  by  the  way  the  program  review  process  handle  non- 
programmatic  action  items  and  minor  issues?  What  issues  that  should  have  been 
handled  through  this  process  were  not?  Why  not? 

•  How  does  the  DHS  program  review  process  handle  issues  that  required 
technical  analysis  prior  to  estimating  the  resources  needed?  What  issues  that 
should  have  been  handled  through  this  process  were  not?  Why  not? 

(3)  Management  of  issue  teams,  e.g.,  conducting  analysis,  developing  products 

•  Do  you  think  your  issue  team  provided  an  objective  view  of  your  issue  to  the 
PRB? 

•  How  much  of  your  time  was  spent  discussing  issues  of  a  technical  nature?  Do 
you  think  you  were  prepared  to  lead  the  technical  discussions  for  your  issues? 
What  could  management  have  done  to  prepare  you  better?  Do  you  think  you 
will  be  prepared  to  lead  the  technical  discussions  for  your  issues  next  year?  If 
not,  what  is  standing  in  your  way? 

•  Issue  team  leads:  Do  you  feel  that  sharing  the  presentation  of  the  PRB  with 
cross-cutting  partners  helped  or  hindered  the  presentation?  Do  you  feel  that 
PA&E  should  be  sole  presenter?  Do  you  feel  the  component  should  be  sole 
presenter? 

•  How  much  time  was  spent  in  issue  team  meetings  deciding  what  the  issue  was? 
How  helpful  was  your  team  in  formulating  the  issue  and  developing  the 
decision  brief? 

•  How  successful  was  your  team  in  developing  metrics?  What  were  obstacles  that 
you  encountered?  Provide  any  examples  of  successful  metrics. 

•  What  issues  were  eliminated  by  the  issue  teams?  Did  these  go  to  the  appeals 
process? 

•  How  involved  were  non-stakeholding  component  organizations  in  your  issue 
team?  What  is  the  appropriate  level  of  involvement?  How  can  PA&E  encourage 
the  component  organizations  to  participate? 

(4)  Coordination  with  budget  review 

•  What  problems  did  you  encounter  with  the  concurrently  conducted  program  and 
budget  reviews?  How  could  these  problems  have  been  avoided? 

•  Should  budget  analysts  attend  the  issue  team  meetings? 

•  How  involved  should  PA&E  be  with  budget  review? 

(5)  Resource  Allocation  Decisions 

•  To  what  extent  were  the  issues  presented  by  your  team  to  the  Program  Review 
Board  funded  in  the  RAD  by  more  than  50  percent  of  requested  funds? 

•  How  well  were  program  and  budget  review  decisions  coordinated  to  construct 
the  RAD? 

•  What  were  obstacles/problems  in  writing  the  RAD  decision  language? 
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ABBREVIATIONS 


CBP 

CFO 

DHS 

DPA&E 

FY 

FYHSP 

ICE 

IDA 

IED 

IPG 

IT 

OCFO 

OMB 

PA&E 

PPBE 

PRB 

RAD 

RAP 

USCG 


[U.S.]  Customs  and  Border  Protection 
Chief  Financial  Officer 
Department  of  Homeland  Security 
Director,  Program  Analysis  and  Evaluation 
fiscal  year 

Future  Years  Homeland  Security  Program 

Immigration  and  Customs  Enforcement 

Institute  for  Defense  Analyses 

improvised  explosive  devices 

Integrated  Planning  Guidance 

information  technology 

Office  of  the  Chief  Financial  Officer 

Office  of  Management  and  Budget 

Program  Analysis  and  Evaluation 

Planning,  Programming,  Budgeting,  and  Execution 

Program  Review  Board 

Resource  Allocation  Decision 

Resource  Allocation  Plan 

United  States  Coast  Guard 
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